Electronic Device Displays an Image of an Obstructed Target

ABSTRACT

An electronic device determines information about a target and provides the information to another electronic device that has an obstructed view of the target. The other electronic device displays an image of the target with an orientation and a location of the target.

BACKGROUND

In order to gather information about a target, an electronic deviceshould be within a predetermined range of the target and have anunobstructed view of the target. When the electronic device is outsideof this range or the target is obstructed, the electronic device cannotgather information about the target or view the target.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a method to display an image of a target at a placementlocation with an electronic device in accordance with an exampleembodiment.

FIG. 2 is a method to display with an electronic device an image of atarget with an orientation in accordance with an example embodiment.

FIG. 3 is a method to display with an electronic device an image of atarget with a size based on a distance from the electronic device to thetarget in accordance with an example embodiment.

FIG. 4 is a method to provide an image of a reference frame of a targetto an electronic device in accordance with an example embodiment.

FIG. 5 is a method to display a target and an area of the target inaccordance with an example embodiment.

FIG. 6A shows a first user with a first electronic device that collectsinformation about a target in accordance with an example embodiment.

FIG. 6B shows a second electronic device with a display that displays afield of view that includes an image or model of the target andadditional information about the target in accordance with an exampleembodiment.

FIG. 7A shows a user wearing a wearable electronic device with a view orline of sight to a target in accordance with an example embodiment.

FIG. 7B shows the target being rotated θ degrees about the Z-axis inaccordance with an example embodiment.

FIG. 7C shows a display of one electronic device of FIG. 7A inaccordance with an example embodiment.

FIG. 7D shows a display of another electronic device of FIG. 7A inaccordance with an example embodiment.

FIG. 8A shows a view of a residential neighborhood with electronicdevices in accordance with an example embodiment.

FIG. 8B shows a display of an electronic device of FIG. 8A in accordancewith an example embodiment.

FIG. 8C shows a display of another electronic device of FIG. 8A inaccordance with an example embodiment.

FIG. 8D shows the display of the other electronic device in accordancewith an example embodiment.

FIG. 9 is an electronic device system or a computer system in accordancewith an example embodiment.

FIG. 10 is an electronic device in accordance with an exampleembodiment.

FIG. 11 is another electronic device in accordance with an exampleembodiment.

SUMMARY OF THE INVENTION

Example embodiments include systems, apparatus, and methods that includeone or more electronic devices that are configured to generate and/ordisplay an image of a target that is obstructed from a view of theelectronic device and/or a user of the electronic device.

Other example embodiments are discussed herein.

DETAILED DESCRIPTION

Example embodiments include systems, apparatus, and methods that includeone or more electronic devices that are configured to generate and/ordisplay an image of a target that is obstructed from a view of theelectronic device and/or a user of the electronic device. The electronicdevice displays the image of the target to coincide with a locationwhere the target exists and with an orientation that matches anorientation of the target as would be seen or sensed from the locationof the electronic device displaying the image of the target.

In one example embodiment, the image of the target appears at a locationin space on a display and/or in the view that corresponds to where thetarget would appear in the view if the target were not obstructed or ifthe target were viewable from the location of the electronic device or auser. Further, the image of the target is rotated or adjusted so itsorientation being displayed matches or emulates a real-time orientationof the target that would appear on the display and/or in the view if thetarget were not obstructed or if the target were viewable from thelocation of the electronic device or a user.

Multiple images of the target can be simultaneously displayed onmultiple different electronic devices that are each located at adifferent geographical location (e.g., located at different distancesfrom the target). One or more of the electronic devices are not able toview, sense, and/or detect the target due to the distance from theelectronic device to the target and/or due to an obstruction between thetarget and the electronic device. The electronic devices shareinformation or data and/or obtain information or data from anotherelectronic device in order to determine one or more of a location of thetarget, an identity of the target, a description of the target, an imageof the target, an orientation of the target, an activity of the target,an area of the target, video and/or audio of the target or area of thetarget, a map or directions to the target, and a model of the target.Based on this information or data, the electronic devices display animage of the target that includes one or more of the location of thetarget, the identity of the target, the description of the target, theimage of the target, the orientation of the target, the activity of thetarget, the area of the target, video and/or audio of the target or areaof the target, a map or directions to the target, and the model of thetarget.

By way of example, this information and/or data can originate from anelectronic device that determines, monitors, detects, tracks, senses,processes, and/or views the target and/or captures images or data of thetarget. For example, the electronic device can be proximate to thetarget (e.g., located within a few meters) or located a greater distancefrom the target (e.g., located more than one hundred meters orkilometers). As another example, one of the electronic devices can belocated on the target, be located with the target, be part of thetarget, or be the target.

The image of the target can include a two-dimensional (2D) image ormodel, a three-dimensional (3D) image or model, a virtual image, and/ora virtual object. This image or model moves to match or emulatereal-time movement of the target. Furthermore, an image displayed ateach of the electronic devices resembles an orientation, size, and/orshape of how the target would appear in a field of view of eachelectronic device or a user of the electronic device if the target werenot obstructed or if the target were viewable from the location of theelectronic device or a user.

FIG. 1 is a method to display an image of a target at a placementlocation with an electronic device.

Block 100 states collect information with a first electronic device of afirst user and determine a target.

By way of example, the target is one or more of a person, an object, athing, or an area; and the first electronic device captures, obtains,records, provides, receives, and/or transmits information about thetarget. For instance, a first user wears and/or uses the firstelectronic device that captures and/or displays an image and/or video ofthe target, determines a distance to the target, determines a locationof the target, captures a view of the target, senses a heat signature ofthe target, determines an identity of the target, or communicates withan electronic device at, with, or near the target.

The target can be determined with information from the first electronicdevice and/or another electronic device. As an example, the firstelectronic device senses a person, recognizes the person with facialrecognition, and selects the person as the target. As another example, ahandheld portable electronic device (HPED) captures images of an areaand transmits these images over a network to a computer system. Thecomputer system analyzes the images and selects a target that is locatedin the images and/or the area. As another example, a user views an areaon a display of the first electronic device and interacts with the firstelectronic device through a user interface to select an object beingdisplayed as the target. As yet another example, the first electronicdevice receives location or coordinate information for a target and thennavigates to this location to determine the target.

Block 110 states collect information with a second electronic device ofa second user and determine one or more objects that obstruct and/orblock the target from being visible to the second user and/or with thesecond electronic device.

By way of example, the first electronic device and/or the secondelectronic device determines, captures, obtains, records, provides,receives, and/or transmits information about the one or more objectsthat obstruct and/or block the target from being visible with the secondelectronic device and/or being visible to a second person using thesecond electronic device. For instance, a user wears and/or uses theelectronic device that captures and/or displays an image and/or video ofthe one or more objects.

The first and second electronic devices can be located at a geographicallocation and near or proximate each other (e.g., located within severalfeet from each other, located several yards or more away from eachother, located within eye sight of each other, located within severalhundred yards of each other, etc.). Alternatively, the first and secondelectronic devices are located at different geographical locations thatare far from each other (e.g., located a kilometer or more apart,located in different cities or states, located in different countries,etc.).

Block 120 states determine a placement location of where the targetwould be visible to the second user and/or the second electronic deviceif the target were not obstructed and/or blocked by the one or moreobjects.

The one or more objects obstruct and/or block the second electronicdevice and/or the second person or user from seeing the target. Forinstance, an object occludes the second electronic device and/or secondperson from being able to capture an image of the target, detect thetarget, see the target, measure the target, or sense the target.

The placement location is the location or place of the target and/orarea of the target where the target exists in a field of view of theelectronic device and/or user (e.g., a user using, holding, or wearingthe electronic device). From the point of view of the second electronicdevice, the placement location is where the target would be visible tothe second electronic device and/or second person if the target were notobstructed and/or blocked by the one or more objects. For example, ifthe one or more objects did not occlude a line of sight of the secondperson, then the second person would be able to see with a naked eye thetarget at the placement location. As another example, if the one or moreobjects did not interfere with the second electronic device, then thesecond electronic device would be able to sense, detect, capture, and/ordetermine the location of or existence of the target.

Block 130 states display, with the second electronic device, an image ofthe target over the one or more objects at the placement location wherethe target would be visible to the second user and/or the secondelectronic device if the target were not obstructed and/or blocked bythe one or more objects.

By way of example, the second electronic device provides or presents animage of the target on, over, with, and/or near the one or more objectssuch that the second user can see or detect the existence of the target.For instance, the second electronic device displays or presents one ormore of two-dimensional (2D) images, three-dimensional (3D) images,virtual images, and/or virtual objects over or in place of the one ormore objects such that the second user can see on the display alocation, a distance, an orientation, a shape, a size, and/or anactivity of the target.

Consider an example in which two users at a geographical location wearwearable electronic devices (WEDs) that capture, record, and transmitinformation concerning the users, the ambient environment, and peopleand objects at the geographical location. This information includescapture images and video within a field of view of the WEDs and/orusers. A first one of the users looks at a window of a nearby buildingand sees a person holding an object. The WED of the first user recordsthis event and determines a location of the person with respect to thefirst user. A second one of the users also looks at the window of thebuilding or looks in the direction of the window but is unable to seethe person holding the object since the second user is located away fromthe first user and sees the window at a different angle. The secondperson is not able to see with a naked eye the person holding the objectsince a portion of the building blocks the view of the second user.Based on the known location, size, and shape of the person and object, avirtual image of the person holding the object is created and providedto the WED of the second user. This virtual image is presented on orthrough the WED so the second user sees the image of the person holdingthe object while the second user looks at the window of the buildingeven though the second person cannot see the actual, real person holdingthe object.

Consider an example in which a first user wears electronic glasses whilelocated in a coffee shop, and a second user wears electronic glasseswhile located outside of the coffee shop. These electronic glassescapture images of the field of views of the users, build 3D images ofthese fields of view, and place these 3D images into a map that showspeople and objects captured with the electronic glasses. The map can besupplemented with additional information, such as images, locations,objects, etc. previously recorded or stored from other users and/orother electronic devices. The first user sees a woman seated at a tablein the coffee shop. The electronic glasses of the first user record thewoman, the table, and the surrounding objects and provide theserecordings to a software program that maps the woman, the table, and thesurrounding objects into an interior map of the coffee shop. Whilelooking at the coffee shop, the second user is unable to see the womanseated at the table and her surroundings since the structure of thecoffee shop blocks the second user from seeing inside the coffee shop.The electronic glasses of the second user, however, overlay 3D images ofthe woman seated at the table such that the second user sees the womanseated at the table and her surroundings. Movements of the woman arerecorded and provided to the map in real-time such that the 3D image ofthe woman moves in synchronization with actual movements of the woman inthe coffee shop.

Consider an example in which a first soldier looks through electronicbinoculars that capture, record, and transmit images seen through theelectronic binoculars. The soldier and electronic binoculars view acombatant (e.g., a person) holding a weapon while located behind a wall.A second soldier attempts to view the combatant through an electronicscope that is mounted to his rifle. The second soldier cannot fire hisrifle at the combatant since the wall partially or fully blocks hisfield of view of the combatant. The electronic binoculars and electronicscope communicate with each other over a peer-to-peer (P2P) network, andthe electronic binoculars provide the electronic scope with combatantinformation. This combatant information includes a location and anidentity of the combatant, a location and an identity of the weapon, 2Dand/or 3D images of the combatant and the weapon, real-time movement ofthe combatant and the weapon. When the second soldier looks through hiselectronic scope, he sees an image of the combatant holding the weaponbehind the wall. Although the second soldier cannot see the combatantand the weapon with a naked eye, he can see an image of the combatantand weapon. This image shows the location, position, and orientation ofthe combatant and the weapon and moves in real time to coincide with themovements of the actual combatant and weapon. The second soldier placesa point of aim of the rifle on the image of the combatant and fires therifle. A bullet from the rifle penetrates through the wall and strikesthe combatant.

Consider an example in which smartphones of users capture information(such as global positioning system (GPS) and video information) andtransmit this information over a network (such as the Internet or aprivate network) to a computer system that builds a map with thecaptured information. A user points a camera in his smartphone toward abuilding, and a display on the smartphone shows images of the interiorof the building even though the interior of the building is not visibleto the user. The smartphone communicates its field of views to thecomputer system that, in turn, retrieves a view from the map of theinterior of the building and transmits this view to the smartphone. Thesmartphone displays the images of the interior of the building as if theinterior were not being obstructed. As such, the user can see inside ofthe building even though the inside of the building is not visible tothe naked eye of the user. Images of the interior of the building appearon the display as if an exterior portion of the building were notblocking the view of the smartphone and/or user.

Consider an example in which a first electronic device captures video ofa person on a bicycle riding on the street. A second electronic device(in communication with the first electronic device) includes a camerathat points in the direction of the street but cannot capture the personon the bicycle because a tree blocks the view of the person on thebicycle. A display of the second electronic device provides an augmentedreality view in which a virtual image of the person on the bicycle isoverlaid on the view of the real world as captured with the camera.

FIG. 2 is a method to display with an electronic device an image oftarget with an orientation.

Block 200 states collect, with a first electronic device at a firstlocation, information about a target located at a location with a firstorientation as seen from the first electronic device at the firstlocation.

By way of example, the first electronic device determines, captures,obtains, records, provides, receives, and/or transmits information aboutthe target. For instance, a first user wears and/or uses the firstelectronic device that captures and/or displays an image and/or video ofthe target.

The first electronic device at the first location collects theinformation with the target being situated at the location with thefirst orientation. This orientation represents a positional view of thetarget from the point of view of the first electronic device. Forexample, the first orientation represents a profile of the target as howthe first electronic device and/or a first user sees or views the targetfrom the first location.

Block 210 states build, from the information from the first electronicdevice, an image of the target having the first orientation as seen fromthe first location of the first electronic device.

For example, an electronic device builds a two-dimensional (2D) and/orthree-dimensional (3D) image or model of the target. This image or modelrepresents, emulates, or copies the target with the first orientation.

Block 220 states determine one or more objects that block and/orobstruct a second electronic device at a second location with a field ofview that would include the target if the target were not being blockedand/or obstructed by the one or more objects.

For example, but for the target being blocked and/or obstructed, thesecond electronic device would see, view, sense, capture, display,and/or include the target at the location of the target. The target maynot be visible and/or detectable to the electronic device for otherreasons, such as the electronic device being too far away from thetarget to sense or detect the target or weather or ambient conditionsprevent the electronic device from sensing or detecting the target.

Block 230 states determine a second orientation for the target at thelocation that the second electronic device would see from the secondlocation if the target were not being blocked and/or obstructed.

The second orientation represents a positional view of the target fromthe point of view of the second electronic device and/or a second user.For example, the second orientation represents how the second electronicdevice and/or the second user would see or view the target from thesecond location. For instance, the second orientation shows how thetarget would be seen with a naked eye of the second user, or how thetarget would be seen with the second electronic device but for targetbeing blocked, but for a far distance to the target, or but forenvironmental conditions that prevent the second user and/or secondelectronic device from seeing, detecting, or sensing the target.

Block 240 states display, with the second electronic device, the imageof the target at the location with the second orientation.

An example embodiment builds the image of the target with informationcollected with one or more electronic devices that view the target,sense the target, and/or receive information about the target (e.g.,receive or obtain information about the target from another electronicdevice or input from a user). For instance, a 2D or 3D image or model ofthe target is built from pictures, video, or other data captured from anelectronic device.

By way of example, an image processor or digital image processorconverts images (such as photos) or video into 3D images, 3D pictures,or 3D models. For example, a 3D modeler converts a photo into a bitmapimage, vectorizes and extrapolates the image, and generates a 3D modelof the photo. As another example, 2D images are combined and/or offsetto generate a stereoscopic image that provides 3D depth perception. Asanother example, a 3D converter converts 2D video into 3D video. Asanother example, a 3D modeler receives a series of photos of a targetfrom multiple views, matches the views, calculates spatial positioning,and generates a 3D point-cloud model that is rotatable three hundred andsixty degrees (360°). As another example, a photo-modeler performs imagebased modeling and close range photogrammetry (CRP) to generatemeasurements from photos and produce 3D models. As another example, a 3Dmodeler retrieves one or more aerial and/or ground photos and executesaerial photogrammetry to generate 2D or 3D models (such as atopographical map or a 3D landscape). As another example, a 3D modelerprocesses video images and renders these images into a rotatabletwo-dimensional image. As another example, a 3D modeler processes photosand/or video and generates one or more of a polygonal model of theimage, a curve model, and a digital sculpture.

The 2D and 3D images represent the target with an orientation from apoint of view or field of view of the viewer and/or the electronicdevice at a particular location and at a particular point in time. Thisorientation would change with movement of the target and/or change whena location of the viewer and/or the electronic device changed. Forexample, another viewer and/or another electronic device located awayfrom the electronic device capturing the information would see or view adifferent orientation of the target. This difference in orientationdepends on a difference in one or more angles from the target to theother viewer and/or the other electronic device with respect to one ormore angles from the target to the location of the electronic devicecapturing the information. Thus, an example embodiment changes anorientation of the target depending on a location of the viewer orviewing electronic device with respect to the target. Changing theorientation presents the image of the target as the target would be seenor viewed if they were not blocked and/or obstructed.

For example in an X-Y-Z coordinate system, an image of target is rotatedabout one or more of the X-axis, Y-axis, and Z-axis to change from oneorientation to another orientation. For instance, a first electronicdevice at coordinate location (X1, Y1, Z1) captures an image of a personat a coordinate location (X2, Y2, Z1) and with a first orientation. Thefirst electronic device builds a 3D image of this person having thefirst orientation. A second electronic device at coordinate location(X3, Y3, Z1) attempts to view the person at location (X2, Y2, Z1), butan object blocks the second electronic device from viewing the person.The second electronic device displays the 3D image of the person at thecoordinate location (X2, Y2, Z1) and with a second orientation. Thissecond orientation is derived from the 3D image based on a change ordifference between coordinate location (X1, Y1, Z1) and coordinatelocation (X3, Y3, Z1). For instance, a geometric and/or trigonometricevaluation of this difference reveals that coordinate location (X3, Y3,Z1) is located along a line in the X-Y plane that is positive forty-fivedegrees (45°) away from coordinate location (X1, Y1, Z1). The 3D imageof the person captured from coordinate location (X1, Y1, Z1) is rotatedin the X-Y plane negative forty-five degrees (45°) and displayed on thesecond electronic device with this rotated orientation. Alternatively, anew 3D image is built from the point of view of coordinate location (X3,Y3, Z1). In both scenarios, the second electronic device displays a viewor orientation of the 3D image of the person that matches the view ororientation of the person that the second electronic device would see,sense, or view if the object did not block the second electronic devicefrom seeing, sensing, or viewing the person.

Information collected from or with one or more electronic devicesassists in determining what orientation to build and/or present to whichelectronic device and where placement locations of images are fordifferent electronic devices and/or users. By way of example, thisinformation includes, but is not limited to, one or more of a GlobalPositioning System (GPS) location, a distance from one electronic deviceto another electronic device, a distance from an electronic device tothe target, a direction of a view (such as a direction of a field ofview, a point of aim, a line of sight, a line of departure, a head of auser, an eyesight of user, and a gaze of a user), an angle or differencein angle between two electronic devices and/or an electronic device andthe target.

A human and/or one or more electronic devices can determine thedirection of a view of a user and/or an electronic device with respectto the target. For example, instruments to measure the direction of theview include, but are not limited to, a compass, a magnetometer, aheading indicator, an inclinometer, a gyroscope, an accelerometer, asensor, or other electrical device to determine direction. Furthermore,triangulation and/or trigonometric identities or functions can be usedto calculate distances, angles between two or more objects and/or two ormore points, locations of targets with respect to different electronicdevices, placement locations, orientations, etc. For example, a laserdetermines a distance from a weapon to the target, and a compassdetermines a direction of a point of aim of the weapon on the target. Anadjustment to an orientation of an image of the target is based onknowing this distance and direction.

By way of example, the second electronic device provides or presents animage of the target on, over, with, and/or near the one or more objectssuch that the second user can see or detect the existence of the target.For instance, the second electronic device displays one or more of 2Dimages, 3D images, and/or virtual images over or in place of the one ormore objects such that the second user can see on the display alocation, a distance, an orientation, a shape, a size, and/or anactivity of the target.

The first user at the first location and the second user at the secondlocation can view the target with one or more different orientations. Byway of example, these orientations include, but are not limited to, theorientation from the point of view or location of the first user and/orfirst electronic device, the orientation from the point of view orlocation of the second user and/or second electronic device, anorientation from the point of view or location of the another userand/or another electronic device, a movable, variable, and/or adjustableorientation (e.g., a user being able to move or set an orientation toview the target), a preset or predetermined orientation (e.g., anorientation that a user or electronic device establishes or selects), arandom orientation, an orientation based on a previously selected orviewed orientation, a favorite or preferred orientation, and anorientation based on a position, orientation, and/or location of a useror electronic device with respect to the target.

Consider an example in which a first user wears a first WED thatcaptures an image of a person in a building while standing in a doorwaythat leads to an alley.

The first user stands in the alley and in front of the person such thatthe first WED captures a frontal image of the person standing in thedoorway and facing the alley. A second user located in the buildingwears a second WED that communicates with the first WED. The second userstands in a room that is directly behind the person but a wall blocksthe second person from seeing the person standing in the doorway. Afield of view of the second user would include a view of the back of theperson standing in the doorway but for the wall blocking the secondperson and second WED from viewing the person. A 3D image of the personstanding in the doorway is built from the information collected with thefirst WED. This 3D image is built from the front orientation of theperson standing in the doorway since this front orientation of theperson is what the first WED sees and captures. The second WED andsecond user, however, are located behind the person and would not seethe frontal orientation of the person, but instead would see a backorientation of the person. The first and second users would see adifferent orientation of the person standing in the doorway since theyare located at different locations with respect to the person. In orderto compensate for the difference in location between the first andsecond users, the second WED is presented with a different orientationof the person. For example, the second WED displays the back orientationof the person standing in the doorway since this is the orientation thatthe second user or second WED would view or see if the person werevisible and not obstructed from the wall blocking the second person andsecond WED from viewing the person.

FIG. 3 is a method to display with an electronic device an image of atarget with a size based on a distance from the electronic device to thetarget.

Block 300 states obtain an image of a target.

For example, an electronic device determines, builds, retrieves,transmits, receives, and/or processes the image of the target.

Block 310 states determine a distance from an electronic device to thetarget.

An electronic or mechanical device measures a distance from theelectronic device to the target. For example, a laser rangefinderdetermines a distance to the target. As another example, a mil dot scopeor milliradian scope provides information to determine a distance totarget. As another example, GPS coordinates or satellite positioninformation provides a distance to the target. As yet another example, acamera determines a distance to the target. As yet another example, auser provides or inputs a distance to the target. As yet anotherexample, a radio transmitter and receiver use radio waves orelectromagnetic waves to determine a distance to the target.

The electronic or mechanical device used to measure distance can be aseparate device (such as a standalone device) or device integrated withor attached to another electronic device (such as a weapon, aprojectile, or another electronic device). For example, electronicand/or mechanical devices in a bow or a firearm determine a distance tothe target. As another example, electronic and/or mechanical devices inone or more of a wearable electronic device (WED), handheld portableelectronic device (HPED), computer, server, and a satellite determine adistance to the target.

Block 320 states determine, based on the distance, a size of the targetto display with the electronic device.

By way of example, image size is inversely proportional to objectdistance and directly proportional to focal length. As a distancebetween a user and the target increases, the image size of the targetdecreases. As the distance between the user and the target decreases,the image size of the target increases. As another example, the size ofthe target is relates to a distance of the electronic device to thetarget. Targets are displayed with a first size given distances in afirst range; targets are displayed with a second size given distances ina second range; targets are displayed with a third size given distancesin a third range; etc. As another example, targets that would not bevisible with a naked eye are displayed with a predetermined size. Asanother example, the size of the displayed target is proportional to orrelates to the distance of the electronic device to the target.

Block 330 states display, with the electronic device, the image of thetarget with the size based on the distance.

The electronic device can display, provide, present, or project theimage of the target with or without magnification of the target. Forexample, the image of the target appears on the display with a size thatis equivalent to a size of the target as seen from the location ordistance of the viewer or electronic device viewing the target.

A size of the target gives a clue or indication of the distance from theviewer to the target. For example, an automobile that is fifty meters(50 m) from a viewer has a size that is much different than thisautomobile when it is one hundred meters (100 m) from the viewer. Basedon a size of the automobile in the field of view, the viewer canestimate a distance from the viewer to the automobile.

Consider an example in which a user wears electronic glasses in alibrary while waiting to meet a friend. An electronic device of thefriend communicates information of the friend to the electronic glasseswhile the friend walks to the library. By way of example, thisinformation includes a GPS location of the friend, an image of thefriend, an orientation of the friend, a direction in which the friend ismoving and/or facing, a rate of movement (e.g., a speed or velocity ofthe friend), images of an area of the friend, audio and/or video at ornear the friend, an activity of the friend, images of other electronicdevices capturing the friend, etc. The user's electronic glasses displayan indication in a field of view of the user to show the user where thefriend is located with respect to where the user is located. Forinstance, a visual indication shows the user in which direction to lookto see an image of the friend with the image being displayed at thelocation where the friend is actually located. Without the electronicglasses, the user would not be able to see the friend walking to thelibrary since the structure of the library and objects located outsideof the library obstruct and/or block the user from seeing the friend.The electronic glasses, however, provide a moving image of the friend asthe friend walks to the library. This image appears in the field of viewof the electronic glasses when the user looks in the direction where thefriend is located. The image also includes an area adjacent to thefriend, such as a portion of the sidewalk or outdoor environment nearthe friend. This adjacent area provides a frame of reference or locationinformation as to where the friend is currently located and/or whatactivity the friend is currently performing. For instance, the user sees3D images or video of the friend, sidewalk, street, and nearby tree.From these images, the user sees that the friend is located on thesidewalk and walking toward the library. The user sees a life-like imageof his friend and sees what his friend is doing while situated in thelibrary.

Consider an example in which soldiers equipped with electronic devicesare dispersed in a large office building while searching for acombatant. A first soldier sees the combatant in a room 520 on the fifthfloor and transmits an image of the combatant at this location to theother soldiers. A second soldier is also located on the fifth floor butis forty meters (40 m) away in another room. When the second soldierlooks toward room 520, the second soldier sees a 3D image of thecombatant. A size of the 3D image of the combatant emulates a size ofthe combatant that the soldier would see while looking to room 520 ifhis field of view and line of sight to the combatant were notobstructed. The size of the combatant gives an indication to the soldieras to how far away the combatant is from the soldier. The 3D image alsoshows a portion of room 520 in order to indicate a location orenvironment in which the combatant is situated. A third soldier islocated on the third floor and in a room that is located under room 520.When the third soldier looks upward toward the location of room 520, thethird soldier sees a 3D image of the combatant. The 3D image of thecombatant shows a view or orientation of the combatant that emulates theorientation of the combatant that the soldier would see while lookingupward to room 520 if his field of view and line of sight to thecombatant were not obstructed.

Consider further the example above in which the soldiers are dispersedin the office building. The 3D image of the combatant displayed to thethird soldier shows a view as seen from underneath the combatant. Forexample, the third soldier sees the bottom of the feet of the combatantwhile the combatant stands in room 520. This view, however, does notshow a face of the combatant or whether the combatant is holding aweapon. The third soldier switches views to see the view being providedto or the view from the point-of-view of the first soldier. The view ofthe first soldier shows a more complete profile of the combatant andincludes a face of the combatant and an image of a weapon that thecombatant is holding. The second soldier also wants to change a view ofthe combatant. Instead of switching views with another soldier, thethird soldier interacts with his electronic device to rotate the 3Dimage of the combatant being displayed to the third soldier. Thisrotated image changes the orientation of the combatant and provides thethird soldier with a three hundred and sixty degree (360°) rotating viewof the combatant.

Consider further the example above in which the soldiers are dispersedin the office building. An electronic device of the first soldiercontinues to capture images of the combatant while the combatant islocated in room 520 and transmits or provides these images to theelectronic devices of the other soldiers. When the second soldier looksin the direction of room 520, the second soldier sees an image of thecombatant that moves in real-time with movements of the combatant. Assuch, the second soldier can view a real-time image, location, distance,and orientation of the combatant. When the second soldier moves hisfield of view or line of sight away from room 520, the 3D image of thecombatant disappears or changes (such as minimizes or reduces on thedisplay, moves to another location on the display, changes to a newwindow, or is replaced with another image, such as an image indicating alocation and/or presence of the combatant).

Consider further the example above in which the soldiers are dispersedin the office building. A fourth soldier with an electronic device islocated on the seventh floor at an opposite side of the office buildingfrom the combatant. A blinking light appears on a display of hiselectronic device. This light indicates the discovery of the combatantand also provides an indication or direction where the fourth soldiershould look to see the combatant. The fourth soldier looks toward room520 on the fifth floor, but the 3D image of the combatant displayed onhis electronic device is small since the fourth soldier is locatedforty-five meters (45 m) from room 520. The fourth soldier interactswith his electronic device to enhance or magnify the displayed view orimage of the combatant. This enhanced or magnified view or image of thecombatant provides more detail and emulates a view as if the fourthsoldier were located three meters away from the combatant.

Consider further the example above in which the soldiers are dispersedin the office building. A fifth soldier is located away from thebuilding and has no frame of reference for where room 520 is located. Anelectronic device of the fifth soldier displays a virtual image of thecombatant holding a virtual object of a firearm in a virtual room shownas room 520. Based on this information, the fifth soldier determinesthat the combatant is armed and located in a room.

FIG. 4 is a method to provide an image of a reference frame of a targetto an electronic device.

Block 400 states determine a target at a location to observe and/ortrack.

An electronic device and/or a human determines the target at thelocation to observe and/or track. For example, selection of the targetis based on one or more of facial recognition or identification of theperson, recognition or identification of the object, gaze tracking ofthe person and/or the user, eye tracking of the person and/or the user,length of time a user looks at the target, user input to an electronicdevice (such as voice input, gesture input, or input through a userinterface), the location of the target, input from an electronic device,input from a third party, and activity or movement of the target.

Block 410 states determine information about a reference frame for thelocation of the target being observed and/or tracked.

The information about the reference frame for the location includes oneor more of a GPS location or coordinates of the location, a descriptionof the location, an address of the location, an image and/or video ofthe location, an image and/or video of surrounding or adjacentlocations, an image and/or video of a path a user traveled to arrive atthe location, aerial and/or satellite views or images of the locationand/or surrounding locations, a map of the location and/or surroundinglocations, rendered images of the location and/or surrounding locations,and information downloaded and/or obtained from the Internet, a website,or a database.

Block 420 states generate an image of the reference frame for thelocation of the target being observed and/or tracked.

For example, the image includes photographs, video, audio, 2D images, 3Dimages, graphics, models, and displayed data and/or information.

Block 430 states provide, to an electronic device, the image of thereference frame for the location of the target being observed and/ortracked.

For example, the image of the reference frame is stored, transmitted,processed, displayed, or provided to the electronic device.

The reference frame assists a user in determining a perspective or aframe of reference for a location or activity of the target and/or animage or model of the target. For example, when an electronic device isaimed or pointed in a direction of the target, an image of the targetdisplays on the electronic device even though the target is too far awayfor the electronic device to capture. This displayed image may notprovide sufficient information for a user to determine a frame ofreference for where the target is actually located or for what thetarget is actually doing. The image of the reference frame assists inproviding this information.

By way of example, the reference frame information includes, but is notlimited to, an aerial view of the target, an enlarged view of thetarget, a view of the target and surrounding area, prior activity of thetarget, movements or locations of the target before arriving at thecurrent location or before commencing the current activity, locationsand activities of people, animals, and/or objects near the target, a mapof an area of the target, historical information, and other informationto assist a user and/or electronic device in obtaining a frame ofreference for or about the target, a location of the target, an activityof the target, persons and/or objects associated with the target, etc.

Consider an example in which an image of John transmits to an electronicdevice of John's parents. The image shows John working on a circuitboard at a workstation. John's parents, however, cannot determine fromthe displayed images where John is located or why he is working on thecircuit board. Images of a reference frame then display on theelectronic device. These images of the reference frame show John islocated in a large airplane hangar with a hundred other people who arebusy building a jetliner. John is working on a circuit board thatrelates to the construction of the jetliner. John's parents now have aframe of reference for where John is located and what John is doing.

Consider an example in which police officers travel to an area to find acriminal suspect. An electronic device of one police officer capturesimages of people at the area and identifies the criminal suspect withfacial recognition software. Instead of apprehending the criminalsuspect, the police officer requests the assistance of other policeofficers in the area. These other police officers, however, aredispersed throughout the area and are unaware of the location of thecriminal suspect. Based on captured images and satellite images, theelectronic device generates a short video that shows the location of thepolice officer, the location of the criminal suspect, and a map showinghow each police officer can travel to the location of the criminalsuspect. This video plays on electronic devices of the other officers sothey can see where the police officer and criminal suspect are located.In addition, the electronic device of the police officer captures videoof the criminal suspect at the location, and this video is rendered orgenerated 3D images or video that are transmitted to a map and/orelectronic devices of the other police officers. When a police officerlooks at the map of the location of the criminal suspect or when apolice officer turns his or her head to look in a direction of thelocation of the criminal suspect, the 3D images or video display to thelooking police officer. In this manner, the looking police officer has areference frame and can see real-time activity and location of thecriminal suspect.

Consider an example in which two friends (John and Paul) decided to meetat a restaurant. John arrives first and waits for Paul who is late. Johnand Paul are members or friends on a social network that enables thesetwo individuals to share real-time location information about eachother. John interacts with his electronic glasses to request a locationof Paul who, unknown to John, is sitting in a taxicab in transit to therestaurant. John's electronic glasses display a visual indication forJohn to look in a North East direction. Paul's electronic glassescapture images or video of him leaving his apartment, hailing a taxicab,entering the taxicab, and his current field of view while sitting in thetaxicab. John moves his head to the indicated direction on the displayand sees 3D imagery of Paul sitting in a taxicab. John sees what Paulsees or what Paul's electronic device has captured or is capturing.Images in John's field of view are 3D transparent line drawings that aresuperimposed over or onto his field of view through the electronicglasses. This imagery includes a short frame of reference to show thecurrent activity and location of Paul. This frame of reference includesimagery of Paul hailing the taxicab and entering into the taxicab. Johnfurther sees Paul sitting in the taxicab as it travels on a nearby roadtoward the restaurant. Based on the size of the taxicab in his field ofview, John estimates that the taxicab is turning into the parking lot ofthe restaurant and is one hundred yards away.

Consider an example in which a soldier views, through an electronicscope mounted to a rifle, a combatant with a weapon on a balcony of abuilding. The electronic scope determines information about thecombatant that includes an identity of the individual, an identity ofthe weapon, a distance to the combatant, a GPS location of the combatantand the soldier, prior criminal activity or history of the combatant,capture or kill instructions for the identified combatant, a location ofthe combatant in the building, blueprints and/or a map of the building,a placement and/or location of the combatant on the map, and video ofthe combatant. The electronic device further visually highlights oroutlines the combatant on its display and tracks movement of thecombatant. Real time images of the combatant captured with theelectronic scope are transmitted to electronic devices of other soldiersand a command center with a commanding officer. In order to provide areference frame for a location of the combatant and/or soldier, theelectronic scope (or another electronic device in communication with theelectronic scope) generates imagery of the reference frame to assist theother soldiers and commanding officer in quickly ascertaining a locationand activity of the combatant and/or soldier. For instance, this imageryincludes one or more of images of a wide angle shot of the buildingtaken from successively farther locations away to show a close-up of thebuilding and balcony, a medium shot of the building, a far shot showingthe entire building, a street map or aerial view with the buildinghighlighted, and an address of the building. This imagery plays ondisplays of the other soldiers and commanding officer and then isfollowed with real-time imagery of the combatant being captured with theelectronic scope. The imagery also includes a map marked with thelocation of the combatant.

In an example embodiment, different reference frames and/or differentviews are provided to different users based on, for example, an identityof a user, a current or previous location of a user, access privilegesof a user, a relationship with the user, a rank or status of a user,knowledge of a user, activity of a user, an electronic device of a user,a distance of the user from the target, etc.

Consider an example in which John has an electronic device that capturesand records his field of view. John desires to share (with two friendsSimon and Paul) images of his friend (Alice) who is located in his fieldof view. Alice and John sit across from each other on benches in a park.Simon is also located nearby in the park, but Paul is located severalmiles away in his apartment. Based on the proximity of Simon to Johnand/or Alice, John's electronic device sends Simon's electronic deviceimagery of a close-up of Alice sitting on the park bench. Other imagesin John's field of view are removed from the imagery sent to the Simon'selectronic device. Simon does not need to see the surrounding images ofAlice since Simon is also located in the park and can determine theframe of reference for the location from the close-up image of Alice.Based on the far distance of Paul to John and/or Alice, John'selectronic device sends Paul's electronic device imagery of Alicesitting on the park bench along with additional imagery. This imageryincludes images of Paul's field of view that surround Alice, imagesPaul's electronic device captured while entering the park beforecapturing the images of Alice, and images of a sign that shows the nameof the park. These additional images are sent to John's electronicdevice since John is not aware that Paul is located in the park. Theseadditional images enable John to quickly determine a frame of referencefor the location of the images of Alice being streamed to his electronicdevice.

FIG. 5 is a method to display a target and an area of the target.

Block 500 states determine, from information collected with a firstelectronic device, a target and an area of the target.

By way of example, the area of the target includes, but is not limitedto, an area around the target, near or proximate or adjacent the target,encompassing the target, touching the target, with the target, enclosingthe target, accompanying the target, above the target, below the target,surrounding the target, providing a frame of reference for the target,and associated with the target. Further, the area can include one ormore of a person, a thing, an object, and space (including occupied andunoccupied space).

By way of further example, a determination is made of a boundary orperimeter of the target. This boundary or perimeter defines a locationof the target. The area of the target includes an area that surroundsthe boundary or the perimeter. For instance, if the target is a person,then the area of the target could be a chair in which the person issitting, a car that the person is driving, a room in which the person islocated, a hill that the person is climbing, etc. Furthermore, the areaof the target can be a predetermined or specified distance or area. Forinstance, if the target is a person, then the area of the targetincludes everything within a certain distance from the target (e.g., anarea within one foot, within two feet, within one yard, within twoyards, within three yards, etc.).

The area of the target can also be defined according to content orcontext of the area. For example, the area of the target includes orextends to weapons near or with the target, drugs near or with thetarget, people near or with the target, vehicles near or with thetarget, ammunition near or with the target, electronic devices near orwith the target, etc. As another example, the area of the target isbased on an activity of the target. For instance, if the target is aperson, then the area of the target includes a bicycle that the personis riding, a car that the person is driving, a book that the person isreading, an electronic device that the person is holding, an animal thatthe person is walking, a third party with whom the person is talking, afirearm that the person is carrying or firing, a field where the personis walking, etc.

The area of the target can also be determined based on a distance, suchas a distance of a user and/or an electronic device from the target. Forexample, an area of the target increases as a distance between thetarget and the first electronic device increases. As another example, anarea of the target decreases as a distance between the target and thefirst electronic device decreases. As another example, an area of thetarget increases as a distance between the target and the firstelectronic device decreases. As another example, an area of the targetdecreases as a distance between the target and the first electronicdevice increases.

The area of the target can also be based on a size of the target. Forexample, an area of the target increases as a size of the targetincreases. As another example, an area of the target decreases as a sizeof the target decreases. As another example, an area of the targetdecreases as a size of the target increases. As another example, an areaof the target increases as a size of the target decreases.

The area of the target can also be based on an amount of space thetarget occupies in the field of view of the user and/or electronicdevice. For example, an area of the target increases as an amount ofspace in the field of view that the target occupies increases. Asanother example, an area of the target decreases as an amount of spacein the field of view that the target occupies decreases. As anotherexample, an area of the target decreases as an amount of space in thefield of view that the target occupies increases. As another example, anarea of the target increases as an amount of space in the field of viewthat the target occupies decreases. As yet another example, if a targetoccupies X percent (X %) of the field of view (where X is a positivenumber between 0 and 100), then an area of the target occupies apercentage of the field of view based on a function of X. For instance,the function of X is X times N (where N is a positive number), X dividedby N, X plus N, X minus N, etc.

The target and the area of the target can also be based on apredetermined size or proportion of a field of view of the user and/orelectronic device. For example, the target and/or area of the target aresized to occupy a certain percentage of a field of view of an electronicdevice (such as occupy 5% of the field of view, 10% of the field ofview, 15% of the field of view, 20% of the field of view, 25% of thefield of view, etc.).

Block 510 states provide the target and the area of the target to asecond electronic device.

For example, the first electronic device transmits the target and/orarea of the target to a second electronic device (such as a WED, WEG,HPED, computer, server, electronic scope, network location, electronicbinoculars, smartphone, network, memory, database, etc.). As anotherexample, the target and/or area of the target are stored to memory orprovided to a network location and retrieved or received with the secondelectronic device.

Block 520 states display the target and/or the area of the target.

The target and/or the area of the target are displayed on the firstelectronic device, the second electronic device, and/or anotherelectronic device with a display.

Consider an example in which an electronic device of a first soldierviews a target (e.g., a combatant standing in a bunker) located fiftymeters (50 m) from the first soldier. The electronic device transmits anotification of the combatant to a second soldier who wears electronicglasses and is located one kilometer from the combatant. The secondsoldier is too far way to see the combatant with a naked eye so a sizeof the target and area of the target (i.e., the combatant and thebunker) is enlarged. When the second soldier turns his head and gazeshis eyes in the direction of the target, the electronic glasses displayan enlarged 3D image of the combatant standing in the bunker. This 3Dimage occupies about 5%-10% of the display space of the electronicglasses and includes a visual indication of the location and distance tothe target. For instance, the electronic glasses display the followinginformation when the second soldier looks in a West direction of twohundred and seventy degrees (270°): “Target Heading: 270°, TargetDistance: 1 km”. When the second soldier turns his head away from thetarget (e.g., his field of view no longer includes the target in theWest direction), the 3D image of the target disappears from the display.

In an example embodiment, the target and/or area of the target isviewable when the user and/or electronic device is directed to alocation of the target and/or area of the target. This provides the userand/or electronic device with a location and/or direction of the targetand/or area of the target.

An example embodiment determines an overlap in different fields of viewfrom different electronic devices and confirms or adjusts an accuracy ofinformation received from the different electronic devices. Thisconfirmation or adjustment can assist in determining a level or amountof confidence of information being determined.

Consider an example in which two electronic devices collect informationabout a target in their respective fields of view. Image analysis ofthis information reveals that an overlap area exists between a field ofview of the two electronic devices. A comparison is made of theinformation collected with the first electronic device in the overlaparea with the information collected with the second electronic device inthe overlap area. A determination is made, based on this comparison, anaccuracy of information in the overlap area. Displays of the first andsecond electronic devices display the information and the accuracy ofthe information to users of the electronic devices.

Consider further the example in which the two electronic devices collectinformation about the target in their respective fields of view. Eachelectronic device determines one or more of a distance to the target, anidentity of the target, a location of the target, an orientation of thetarget, an activity of the target, movement of the target, and otherinformation discussed herein. The information determined with each ofthese two electronic devices is compared to determine an accuracy of theinformation. For example, both electronic devices provide images thatconfirm an identity of the target is a person named R. Louis. As anotherexample, information from the first electronic device determines thatthe target includes a person that stands five feet ten inches tall (5′10″), and information from the second electronic device determines thatthe target includes a person that stands five feet eleven inches tall(5′ 11″). A determination is made that the target has a height between5′ 10″ and 5′ 11″. As another example, information from each of theelectronic devices determines that the target includes a person holdinga weapon. This information from two independent sources confirms thatthe target is armed and dangerous.

FIG. 6A shows a first user 600 with a first electronic device 610 thatcollects information about a target 620 located on a second floor of abuilding 630. The first user 600 and/or first electronic device 610 havean unobstructed view or line of sight 640 to the target 620. A seconduser 650 with a second electronic device 660 has an obstructed view orline of sight 670 to the target 620 since a side 675 of the building 630blocks or impedes the second user 650 and/or second electronic device660 from viewing, sensing, detecting, and/or perceiving the target 620.

The first electronic device 610 shares information with the secondelectronic device 660 so the second electronic device 660 can display orprovide information about the target. By way of example, thisinformation includes an image or model of the target, a location of thetarget, an orientation of the target, an object with the target, anactivity of the target, a direction to the target, a distance to thetarget, a view of the target, and other information determined with thefirst electronic device and/or obtained from another electronic device.

FIG. 6B shows the second electronic device 660 with a display 670 thatdisplays a field of view that includes an image or model 680 of thetarget and additional information 690 about the target. By way ofexample, this additional information 690 includes an identification (Id)or name of the target (shown as “R. Louis”), a distance (D) from thesecond electronic device to the target (shown as “87.5 m”), a locationof the target on the second floor of the building 630 (shown as “2ndFl.”), and a compass or heading direction to the target (shown as “270°W”). The display 670 as indicates that a location and movement of thetarget are being tracked (shown as “Tracking: On”).

The side 675 of the building 630 blocks the second user 650 and/orsecond electronic device 660 from seeing the target 620. For example, acamera, scope, viewfinder, or other mechanical or electrical viewingdevice on the second electronic device 650 is not able to see or capturea photo, video, or other information of the target 620 since thebuilding obstructs the line of sight. Further, the second user 650 alsowould not be able to view the target with a naked eye since the buildingobstructs a view to the target. The second electronic device 650,however, provides an image or model 680 of target 620 so the user cansee the target.

As shown in FIG. 6A, the first user 600 and first electronic device 610view a front orientation or front profile of the target 620 since thetarget (which is a person) faces the first user and first electronicdevice. Viewing the target with this orientation, the first electronicdevice captures an image, photo, or video of the target and determinesother information (such as an identity of the target with a facialrecognition program, a GPS location of the target, a distance to thetarget, an identity or existence of weapons or other objects with ornear the target with an object recognition program, and an activity ofthe target).

As shown in FIG. 6B, the second electronic device 660 displays a sideorientation or side profile of the target 620 since a side of the target(which is a person) faces the second user and second electronic device.The second electronic device displays an image of the target in anorientation or profile that matches an orientation or a profile of whatthe second electronic device and/or second user would see if the view orline of sight to the target were not obstructed. The image displayed onthe second electronic device also matches a size and shape of the targetthat the second electronic device and/or second user would see if theview or line of sight to the target were not obstructed. Further, aplacement location of the image of the target in the building matchesthe actual physical location in the building where the target islocated.

The image 680 of the target 620 disappears or is removed from thedisplay 670 when a line of sight of the user 650 and/or secondelectronic device 660 does not include the location of where the target620 is situated. For example, when the user 650 and second electronicdevice 660 move the line of sight 670 or field of view away from thebuilding 630, the user and the second electronic device would no longerbe able to see the target even without the obstructions. When thismovement occurs, the target is removed from the display since the targetis no longer in the field of view of the user and the second electronicdevice.

The image 680 of the target 620 remains on the display 670 when the lineof sight 640 or field of view of the user 600 and/or first electronicdevice 610 does not include the location of where the target 620 issituated. For example, when the user 600 and first electronic device 610move the line of sight 640 or field of view away from the building 675,the image 680 of the target 620 remains displayed on the secondelectronic device 660. The second electronic device 660 knows a locationand other information of the target and can track the target (e.g.,using infrared or information received from another electronic device).

FIG. 7A shows a user 700 wearing a wearable electronic device 710 with aview or line of sight 715 to a target 720. Another user 730 wears awearable electronic device 740 with a view or line of sight 745 to thetarget 720. An object 750 blocks or obstructs the view or line of sight745 to the target 720.

FIG. 7A includes an X-Y-Z coordinate system in which the lines of sight715 and 745 are located in an X-Y plane. An angle 755 (shown as θ)exists between the lines of sight 715 and 745 in this X-Y plane.Further, the figure shows a distance A from user 700 and wearableelectronic device 710 to target 720, a distance B from user 700 andwearable electronic device 710 to user 730 and wearable electronicdevice 740, and a distance C from user 730 and wearable electronicdevice 740 to target 720.

For illustration, target 720 is shown as a three dimensional cube withsix sides. Each side of the cube has a different number of dots thatindicates a number of the cube (i.e., one side has one dot, one side hastwo dots, one side has three dots, one side has four dots, one side hasfive dots, and one side has six dots).

The angle (θ) 755 represents an offset, rotation, or difference from theline of sight 715 to the line of sight 745. This angle providesinformation as to the different orientations or profiles visible,sensible, or detectable along the respect lines of sight.

FIG. 7B shows the target 720 being rotated θ degrees about the Z-axis.Given that the original orientation of the target 720 was along line ofsight 715, this rotation determines the amount of adjustment to anorientation for user 730 and electronic device 740. For example, if θ isninety degrees (90°) about the Z-axis, then the difference inorientations between electronic device 710 and electronic device 740 isninety degrees (90°). An image or model built from the perspective alongline of sight 715 would be rotated or adjusted ninety degrees (90°)along the Z-axis to provide an orientation for the view along the lineof sight 745.

In FIG. 7A, target 720 shows three sides of the cube that represent theview of the user 700 or wearable electronic device 710 along the line ofsight 715. In FIG. 7B, target 720 shows three sides of the cube thatrepresent the view of the user 730 or wearable electronic device 740along the line of sight 745.

The user 730 and wearable electronic device 740 are not able to view thetarget 720 since the target is obstructed or blocked with object 750.For instance, the user 730 and wearable electronic device 740 may notknow of the existence or presence of the target since they are unable tosee it. Further, the user 730 and wearable electronic device 740 may notknow the location of the target, the distance to the target, theactivity of the target, the orientation of the target, the image of thetarget (e.g., what the target looks like), or the identity of the targetsince they are unable to see the target.

Wearable electronic device 710 gathers or captures information thatassists the wearable electronic device 740 in determining one or more ofthe existence or presence of the target, the location of the target, thedistance to the target, the activity of the target, movement or activityof the target, the orientation of the target, the image of the target(e.g., what the target looks like), or the identity of the target.

By way of example, the wearable electronic device 710 captures one ormore images of the target 720 and builds a 3D model of the target. Thewearable electronic device 740 displays the 3D model of the target tothe user 730 with an orientation that matches an orientation the user730 and wearable electronic device 740 would see of the target if thetarget were not obstructed or blocked by the object 750. Knowing alocation of the user 730 and the wearable electronic device 740 withrespect to the target 720 and the user 700 and the wearable electronicdevice 710, the 3D model is rotated or built to present an orientationto the user 730 and wearable electronic device 740 that matches theactual, real orientation of the target from their location.

FIGS. 7A and 7C show the electronic device 710 with a display 760 thatdisplays the target 720 to the user 700. The display 760 displays threefaces 770A, 770B, and 770C of the target 720 and provides the user 700with the orientation along the line of sight 715. The user 700 sees thetarget 720 as it appears along the line of sight 715 since the target isnot obstructed with the object 750.

FIGS. 7A and 7D show the electronic device 740 with a display 780 thatdisplays the target 720 to the user 730. The display 780 displays threefaces 770B, 770C, and 770D of the target 720 and provides the user 730with the orientation along the line of sight 745. The user 730 is thusable to see the target 720 along the line of sight 745 at the targetwould appear if the target were not obstructed with the object 750.

FIG. 8A shows a view of a residential neighborhood 800 that includes atarget 810 (shown as a person holding an object) next to a tree 812 onone side of a house 814, an automobile 816, and a group of trees 818 onanother side of the house 814. A handheld electronic device 820 with auser 830 determines information about the residential neighborhood 800in a field of view 840. By way of example, this information includes oneor more of photos, video, images (such as infrared images or othersensed images), audio, GPS locations and distances of objects and peoplein the residential neighborhood, and other information discussed herein.The electronic device 820 moves to determine the information fromvarious angles, positions, locations, lines of sight, and/or fields ofview.

The residential neighborhood includes a second user 850 that standsbetween the house 814 and the group of trees 818. The second user 850looks in the direction of the target 810 but is unable to view thetarget 810 since the house 814 blocks or obstructs a view of the target810.

The residential neighborhood also includes a third user 860 that islocated in a house 862 (see also FIG. 8C) several blocks away from thehouse 814. The third user 860 looks in the direction of the target 810but is unable to view the target 810 since the house 814 and otherobjects in the neighborhood block or obstruct a view of the target 810.

Looking also to FIG. 8B, a portable electronic device 870 with thesecond user 850 includes a display 872 that provides a view of theelectronic device 870 and/or second user 850. Without the assistance ofthe electronic device 870, the second user 850 would be unable to see orview the target 810 since the house 814 is located between the seconduser 850 and the target 810 and thus blocks a view of the target 810.The display 872 of the second electronic device 870 displays the target810 and an area of the target 874 that includes the tree 812 and aground area 876 where the target 874 stands.

The display 872 displays actual images or a 2D or 3D image or model ofthe target 810 and the area of the target 874 on or over the house 814since this is the view that the second user 850 and/or electronic device870 would see if a view of the target were not obstructed with thehouse. A size of the target 810 and objects in the area of the target874 appear on the display 872 and/or to the user 850 with a size basedon a distance from the second user 850 and/or electronic device 870 tothe target 810. For example, their size is inversely proportionally tothe distance to them.

Furthermore, an orientation or profile of the target 810 and the area ofthe target 874 are displayed as the second user 850 and/or electronicdevice 870 would see the target and the area of the target if theseobjects were not obstructed. For example, the target 810 in FIG. 8A hasa forward facing or front profile or orientation to the electronicdevice 820 and user 830. The target 810 in FIG. 8B, however, has a sidefacing or side profile or orientation to the electronic device 870 anduser 850.

The target 810 and the area of the target 874 are visually distinguishedon the display from objects in a line of sight of the second user 850and/or electronic device 870. This visual distinction enables a user todistinguish between objects that are in the line of sight of theelectronic device and/or second user (e.g., a side 878 of the house 814)and objects being displayed that are obstructed and not in the line ofsight of the electronic device and/or second user (e.g., the target 810and the area of the target 874). By way of the example, the objectsbeing displayed that are obstructed and not in the line of sight of theelectronic device are displayed with visual distinction using one ormore of dashed lines, different colors or tones or brightness,highlights, modeled images, 2D images, 3D images, augmented images,indicia, text, and other techniques to visually distinction or identifythe objects being displayed that are obstructed and not in the line ofsight of the electronic device.

In some instances, providing a user and/or electronic device with imagesof the target and area of the target may not be sufficient to provide avisual reference frame for a location and/or activity of the target. Inaddition to displaying the target and/or area of the target, anelectronic device can display a reference frame for the obstructedtarget.

Looking also to FIG. 8C, an electronic device 880 with the third user860 includes a display 882 that provides images or video of a referenceframe for a location and/or activity of the target 810. By way ofexample, this reference frame includes a map 884 of the residentialneighborhood. The map 884 shows that the house 862 of the third user 860is located several blocks away from the house 814 where the target 810is located. The map 884 also includes an address 886 of the house 814and arrows 888 showing directions how to get from house 862 to house814.

Looking also to FIG. 8D, the display 882 of the electronic device 880also displays images, photos, and/or video of the target 810 and an areaof the target 890 that includes images, photos, and/or video of the tree812 and automobile 816. Without the assistance of the electronic device880, the third user 860 would be unable to see or view the target 810since houses (such as house 814 and house 862) and other objects in theneighborhood block a view of the target 810.

The electronic device 880 overlays, provides, or displays on the display882 a view 895 of the real-time view that the first user 830 and/orfirst electronic device 820 sees, detects, captures, determines, and/orsenses in the field of view 840. This view 895 is a snapshot of thefield of view 840 and can include or be the field of view 840 or includeor be a portion of the field of view 840. For example, the view can be afraction or portion of the field of view 840 with some portions of thefield of view being omitted. As another example, the view can includeadditional or more objects or views than the field of view, such asinclude images in a current field of view and images in a previous oranother field of view.

A size and/or shape of the view 895 can vary. For example, FIG. 8D showsthe view with a wiggled, rippled, or waved circular or oval shape, butthe view can have other shapes as well (such as a square shape, acircular shape, a rectangular shape, an oval shape, a polygonal shape,etc.). The closed wiggled, rippled, or waved contour or closed perimeterdistinguishes the view and images in the view from the real or actualview.

Further, the view 895 can be stationary or movable. For example, theelectronic device 880 sets or associates the view at certain coordinatelocation, compass direction, a GPS location, direction of view,direction of gaze or focus, line of sight, field of view, etc. Forinstance, display or show the view when a line of sight or field of viewof an electronic device and/or user is directed toward a location of thetarget or a designated point in space (e.g., a point in space designatedto include the view). For instance, display the view when a user and/orelectronic device views or points to the ground or another designateddirection or location (such as a specific angle, degree, or compassdirection).

Consider an example in which the view 895 appears at a location on asee-through or transparent display. The location coincides with thelocation of where the target is actually located, or the locationcoincides with a designated location for where a user can look or pointthe electronic device to display the view. For instance, display theview when the electronic device and/or user looks, directs, points,tilts to an arbitrary point or location in space, a predetermined pointor location in space, or a designated point or location in space. Theview provides a virtual reality or an augmented reality view thatenhances the actual view with digital images overlaid onto real images.

Consider an example in which John holds his smartphone while he standsoutside of a building waiting for his friend Paul who is located insideof the building. John points a face or side of the smartphone atdifferent locations along his view of the building. When a line of sightor a point of aim of the face or side of the smartphone coincides orincludes the location of Paul, then a display on the smartphone shows areal-time virtual image of Paul. John is thus able to see where Paul islocated inside of the building since the line of sight or the point ofaim of the face or side of the smartphone points to where Paul islocated. The virtual image of Paul disappears when the line of sight orthe point of aim of the face or side of the smartphone no longer pointsto where Paul is located.

By way of example, electronic device 880 is a pair of electronic glassesthat the user 860 wears. When the user 860 looks in the direction of thetarget 810, the display 882 displays an image of the target 810 and areaof the target 890. The target 810 and the area of the target 890disappear or are removed when the user 860 looks in a differentdirection (i.e., away from the target 810).

An electronic device can simultaneously display, track, and/or includemultiple different targets at multiple different locations. Theelectronic device can filter certain targets from being displayed whileselecting other targets to be displayed. A decision as to whether todisplay a target or not to display a target can be based on any one ofvarious factors that include, but are not limited to, a location of thetarget, a distance to the target, a relationship to the target, amembership of the target, an identification or name of the target, anobject with or near the target, an activity of the target, or anotherfactor relating to the target and/or a user. By way of example, anelectronic device includes targets within a geographical radius of fivehundred meters (500 m), includes targets of people that are members to asocial network to which the user also is a member, includes targetswithin a building, includes targets within a city, includes targets in aroom, includes targets with weapons or firearms, includes target with afamilial relation to a user, includes targets identified as a combatantor criminal, includes targets selected by a user or another electronicdevice, includes targets with a personal profile that matches a personalprofile of a user, etc.

Consider an example in which a user belongs to a social network (such asFACEBOOK) and has fifty other users or friends that also belong to thissocial network. Electronic devices of the users or members displayimages, locations, activities, and/or other information discussed hereinof the fifty other users while excluding non-members of the socialnetwork.

Consider an example in which two friends (John and Paul) agree tocollect and share information from their handheld portable electronicdevices (HPEDs). The HPEDs collect and determine information asdiscussed herein. For instance, each HPED determines one or more of alocation of the user, an activity of the user, an orientation or profileof the user, a direction a user is facing, an object a user is holding,a 2D or 3D image of a user, video of a user, and audio of a user. Whenan HPED of John points in a compass direction of where Paul is located,John's HPED shows a real-time image of Paul and other informationdiscussed herein. When an HPED of Paul points in a compass direction ofwhere John is located, Paul's HPED shows a real-time image of John andother information discussed herein. The HPEDs of John and Paul alsodisplay maps with a visual indication of their location on the map(e.g., display a dot, a pin, an arrow, or other indicia to indicate apresent location of the HPED and/or user on the map). When John or Paulactivates the visual indication through a user interface (e.g., touchthe display at the visual indication), then the display shows the imageof the target at the location as discussed herein.

Consider an example in which John and his friends belong to a socialnetwork. When a friend is within a predetermined proximity of John, anelectronic device of John displays a visual indication of the friend. Byway of example, this visual indication instructs John in which directionto look to see an image of his friend, a location of where the friend islocated, a distance to the friend, an image of the friend, or othervisual information discussed herein. Further, John's electronic devicedisplays a map that indicates where each friend is located. Anenlargement of the map at a location of a friend shows images of thefriends as discussed herein. John can navigate from map location to maplocation of friends to see images of what each friend is doing, wherethe friend is located, an area of the friend, objects with which thefriends are interacting, video and audio of the friends, and otherinformation discussed herein.

Consider an example in which soldiers in a platoon have electronicdevices that capture and determine information as discussed herein. Thisinformation is shared in a private peer-to-peer network to enable thesoldiers to locate, track, identify, determine, and display enemycombatants.

In an example embodiment, generation, display, determination, and/ortransmission of images of a target occur in response to a request from ahuman and/or an electronic device. Consider an example in which imagesof a target are stored at a network location and are continually updatedin real-time as new or updated information about the target is received.A camera of an electronic device points in a direction of the target butis unable to view the target due to an object that obstructs the target.The camera captures an image of the object, and object recognitionsoftware identifies the object as an obstruction to viewing the target.The electronic device communicates its location and a point of aim ofthe camera to the network location and transmits a request for real-timeimages of the target. In response to this request, the network locationtransmits the images of the target to the electronic device, and theimages are displayed on the electronic device with the orientation orprofile that the camera would see if the object did not obstruct thetarget.

Consider an example in which two portable and/or wearable electronicdevices determine information about a target at a geographical location.A first one of these electronic devices determines a location and afirst orientation of a target (such as a person holding a weapon) thatare visible with a naked eye of a first user and that are visiblethrough or with the first electronic device. A second one of theseelectronic devices obtains the information determined with the firstelectronic device and hence determines the location of the target. Thetarget is not visible and/or detectable to the second electronic devicesince an object obstructs or blocks a view of the second electronicdevice to the target. For instance, object and/or facial recognitionsoftware determines that the target is not visible at the location witha naked eye of the second user because objects obstruct the second userfrom seeing the target at the location. Based on one or more of thelocations of target, the first electronic device, and the secondelectronic device, a placement location for the target is determined.This placement location is where the target would be visible to thenaked eye of the second user in a field of view of the second user ifthe target were not obstructed by the objects. Analysis of informationcollected with the first electronic device determines a firstorientation of the target at the location. For example, this firstorientation includes a profile, position, or direction of gaze of thetarget from the point of view or field of view of the first electronicdevice. Analysis of the information collected with the first and/orsecond electronic devices also determines a second orientation of thetarget at the location. For example, this second orientation includes aprofile, position, or direction of gaze of the target from the point ofview or field of view of the second electronic device. For instance,this second orientation depicts how the target would be oriented at thelocation to the naked eye of the second user in the field of view of thesecond user if the target were not obstructed by the objects. A 3Dmodeler constructs 3D images of the target and/or an area surroundingthe target. A display of the second electronic device displays the 3Dimages of the target having the second orientation and being located atthe placement location in the field of view of the second user such thatthe 3D images of the target appear in the field of view of the seconduser where the target would be visible to the naked eye of the seconduser in the field of view of the second user if the target were notobstructed by the objects.

Consider further the example above in which two electronic devicesdetermine information about the target at the geographical location. Thesecond electronic device determines a distance from the secondelectronic device to the target. Based on this distance, a determinationis made of a size of the target (including an object the target holds orincludes) that would appear in the field of view of the second user ifthe target were not obstructed by the objects and visible with the nakedeye of the second user in the field of view of the second user. Thesecond electronic device displays the 3D images of the target with thesize of the target that would appear in the field of view of the seconduser if the target were not obstructed by the objects and visible withthe naked eye of the second user in the field of view of the seconduser.

Consider further the example above in which two electronic devicesdetermine information about the target at the geographical location. Anelectronic device determines, from the information collected with thefirst electronic device, a 3D image of an area that is proximate to thetarget. The second electronic device displays this 3D image of the areathat is proximate to the target such that the 3D image of the areaappears with the 3D images of the target in the field of view of thesecond user where the area would be visible to the second user in thefield of view of the second user if the area were not obstructed by theobjects and visible with the naked eye of the second user in the fieldof view of the second user.

Consider further the example above in which two electronic devicesdetermine information about the target at the geographical location. Anelectronic device determines a facial image from the informationcollected and an identity of a person of the facial image. The 3D imagesof the target are drawn to resemble the identity of the person anddisplayed on the second electronic device.

Consider further the example above in which two electronic devicesdetermine information about the target at the geographical location. Thesecond electronic device detects movement from its current field of viewthat includes the location of the target to another field of view thatdoes not include the location of the target. But for the objectsobstructing the target, the target would visible in the current field ofview, but the target is not visible in the other field of view sincethis other field of view does not include the location of the targeteven if the target were not obstructed. In response to detecting themovement of the second electronic device to the other field of view, thesecond electronic device removes the 3D images of the target beingdisplayed through the second electronic device. Removal of the 3D imagesoccurs at a time when the field of view of the second electronic deviceand/or second user no longer includes the location of the target even ifthe target were not obstructed by the objects.

Consider further the example above in which two electronic devicesdetermine information about the target at the geographical location. Anelectronic device analyzes the collected information and determinesmovement of the target. The second electronic device displays movementof the 3D images of the target to coincide in real time with themovement of the real target as captured with the first electronic deviceof the first user.

Consider further the example above in which two electronic devicesdetermine information about the target at the geographical location. Thesecond electronic device detects movement from its current field of viewthat includes the location of the target to another field of view thatdoes not include the location of the target. In response to thisdetection, the second electronic device displays a visual indicationthat indicates a direction of the location in order to visually informthe second user where to look in order to see the 3D images of theperson and the weapon at the location. Based on this visual indication,the second user knows in which direction to turn his head or his gaze inorder to see the 3D images of the person since these images are locatedat the placement location that represents the actual location.

Consider an example of an electronic system that includes a firstelectronic device, a second electronic device, and a computer thatcommunicate with each other over a network (such as a wireless network).A camera with the first electronic device captures an image of an objectand a person in a field of view of a first user wearing or using thefirst electronic device. A camera with the second electronic devicecaptures an image of the object without the person since the person isblocked or obstructed by the object (e.g., located behind the object)and not within a field of view of a second user wearing or using thesecond device. The computer receives or obtains the image of the objectand the person from the first electronic device and receives or obtainsthe image of the object without the person from the second electronicdevice. With this information, the computer creates a three dimensional(3D) image of the person to resemble the person, determines a locationof where the person would appear in the field of view of the second userif the object were not obstructing the person in the field of view ofthe second user, determines an orientation of what profile of the personthat the second user would see in the field of view of the second userif the object were not obstructing the person in the field of view ofthe second user, and transmits the 3D image of the person to the secondelectronic device. A display of the second electronic device displaysthe 3D image of the person at the location and with the orientation inthe field of view of the second user as if the object were notobstructing the person in the field of view of the second user.

Consider further the example above of the electronic system thatincludes the first and second electronic devices and computer thatcommunicate with each other over the network. The second electronicdevice displays the 3D image of the person with a shape, a size, and anorientation that matches a shape, a size, and an orientation of thereal, actual person located behind the object. The shape, the size, andthe orientation match what the second user and/or second electronicdevice would see if the person were not obstructed with the object.

Consider further the example above of the electronic system thatincludes the first and second electronic devices and computer thatcommunicate with each other over the network. The computer moves the 3Dimage of the person to match real time movement of the person locatedbehind the object even though the person is not visible with a naked eyeof the second user in the field of view of the second user. Movement ofthe 3D image correlates or coincides with movement of the real person asdetermined from information collected with the first electronic device.

Consider further the example above of the electronic system thatincludes the first and second electronic devices and computer thatcommunicate with each other over the network. The display of the secondelectronic device removes the 3D image of person when the field of viewof the second user and/or second electronic device moves to a locationthat no longer includes the object. For example, the camera of thesecond electronic device moves such that its field of view no longerincludes the location of the person. The 3D image of the personreappears on the display of the second electronic device when the fieldof view of the second user and/or second electronic device moves back tothe location that includes the person.

Consider further the example above of the electronic system thatincludes the first and second electronic devices and computer thatcommunicate with each other over the network. With the informationreceived from the first and/or second electronic devices, the computercreates a 3D image of a portion of the object that is visible to thefirst user and/or first electronic device but not visible to the seconduser and/or second electronic device and transmits the 3D image of theportion of the object to the second electronic device. The display ofthe second electronic device displays the 3D image of the portion of theobject such that the second user sees the object supplemented with the3D image of the portion of the object that is not visible to the seconduser and/or second electronic device.

Consider further the example above of the electronic system thatincludes the first and second electronic devices and computer thatcommunicate with each other over the network. The display of the secondelectronic device displays a distance from the second user and/or secondelectronic device to the person even though the person is not visiblewith a naked eye of the second user and/or visible to the secondelectronic device in the field of view of the second user and/or secondelectronic device.

Consider further the example above of the electronic system thatincludes the first and second electronic devices and computer thatcommunicate with each other over the network. The computer analyzes theimage of the person from the first electronic device, determines thatthe person holds a firearm pointed in a direction, and transmits the 3Dimage of the person holding the firearm to the second electronic device.A display of the second electronic glasses displays the 3D image of theperson holding the firearm pointed in a direction that matches theperson holding the firearm pointed in the direction even though theperson and the firearm are not visible with a naked eye of the seconduser and/or visible to the second electronic device in the field of viewof the second user and/or second electronic device but are visible witha naked eye of the first user and/or visible to the first electronicdevice.

Consider an example in which a computer system includes one or moreelectronic devices that execute a method. The computer system obtains,from a first electronic device, an image of a person holding a weapon ata location visible in a field of view of a first user wearing or usingthe first electronic device. The computer system creates, from the imageof the person holding the weapon at the location, 3D images of theperson, the weapon, and the location such that the 3D images show theperson holding the weapon at the location. The computer system obtains,from second electronic device, an image of an object that blocks a fieldof view of a second electronic device and/or second user wearing orusing the second electronic device from seeing or sensing the personholding the weapon at the location. A determination is made of aplacement location in the field of view of the second user and/or secondelectronic device where to position the 3D images of the person, theweapon, and the location such that the 3D images of the person, theweapon, and the location at the placement location show the personholding the weapon at the location as the person holding the weapon atthe location would appear in the field of view of the second user and/orsecond electronic device if the object were not blocking the second userand/or second electronic device from seeing the person holding theweapon at the location. A display of the second electronic devicedisplays, over the object in the field of view of the second user and/orsecond electronic device, the 3D images of the person, the weapon, andthe location at the placement location such that the field of view ofthe second user and/or second electronic device shows the person holdingthe weapon at the location as if the object were not blocking the seconduser and/or second electronic device from seeing the person holding theweapon at the location.

Consider further the example in which the computer system includes oneor more electronic devices that execute the method. The computer systemdetermines, from the image of the person holding the weapon at thelocation, an orientation of the weapon and a point of aim of the weaponand further adjusts the 3D images of the person, the weapon, and thelocation to match the orientation of the weapon and the point of aim ofthe weapon. The display of the second electronic device displays, overthe object in the field of view of the second user, the 3D images of theperson, the weapon, and the location such that the field of view of thesecond user and/or second electronic device shows the person holding theweapon with the orientation and the point of aim at the location as ifthe object were not blocking the second user and/or second electronicdevice from seeing the person holding the weapon with the orientationand the point of aim.

Consider further the example in which the computer system includes oneor more electronic devices that execute the method. The secondelectronic device switches views from displaying the 3D images of theperson, the weapon, and the location over the object in the field ofview of the second user and/or second electronic device to displayingreal time images from the first electronic glasses that show the personholding the weapon at the location in the field of view of the firstuser and/or first electronic device.

Consider further the example in which the computer system includes oneor more electronic devices that execute the method. The secondelectronic glasses displays a visual indication that provides a compassdirection of the location of the person holding the weapon with respectto a location of the second electronic glasses. When the secondelectronic glasses move its camera to point in the compass direction,the display of the second electronic glasses displays the 3D images ofthe person, the weapon, and the location.

Consider further the example in which the computer system includes oneor more electronic devices that execute the method. When the secondelectronic glasses move its camera to point away from the compassdirection, the display of the second electronic glasses removes the 3Dimages of the person, the weapon, and the location from being displayed.

Consider further the example in which the computer system includes oneor more electronic devices that execute the method. The display of thesecond electronic glasses displays, on a see-through display, the 3Dimages of the person, the weapon, and the location in a view that has awaved circular shape with a closed configuration.

FIG. 9 is an electronic device system or a computer system 900 thatincludes one or more of the following: a target 910 including or incommunication with electronics or an electronic device 912, a server920, a database 930 or other storage, a handheld portable electronicdevice or HPED 940, a wearable electronic device or WED 950, wearableelectronic glasses or WEG 960, a plurality of electronic devices 970A,970B, and 970C, and one or more networks 980 through which electronicdevices can communicate (such as wirelessly communicate).

FIG. 10 is an electronic device 1000 that includes one or more of thefollowing: a processing unit or processor 1010, a computer readablemedium (CRM) or memory 1020, a display 1030, one or more interfaces 1040(such as a network interface, a graphical user interface, a naturallanguage user interface, and/or an interface that combines reality andvirtuality), a battery or a power supply 1050, wireless communication1060, an image processor or a 3D modeler 1070, and an orientationdeterminer 1080 (such as a system that executes one or more exampleembodiments discussed herein to determine an orientation of a target).

FIG. 11 is an electronic device 1100 that includes one or more of thefollowing: a processing unit or processor 1110, a computer readablemedium (CRM) or memory 1120, a display 1130, one or more interfaces 1140(such as a network interface, a graphical user interface, a naturallanguage user interface, and/or an interface that combines reality andvirtuality), one or more recognizers 1150 (such as object recognitionsoftware, facial recognition software, and/or animal recognitionsoftware), one or more sensors 1160 (such as micro-electro-mechanicalsystems sensor, a motion sensor, an optical sensor, radio-frequencyidentification sensor, a global positioning satellite sensor, a solidstate compass, gyroscope, an accelerometer, and/or a weather sensor), acamera 1170, a global positioning system or GPS 1180, audio 1190 (suchas a microphone or speakers), a distance determiner 1192 (such as alaser, an electromagnetic wave transmitter/receiver, a rangefinder,and/or a camera), a direction determiner or an orientation determiner1194 (such as a compass, a magnetometer, a heading indicator, aninclinometer, a gyroscope, an accelerometer, a sensor, or otherelectrical device to determine direction).

FIGS. 10 and 11 show various components in a single electronic device.One or more of these components can be distributed or included invarious electronic devices, such as some components being included in anHPED, some components being included in a server, some components beingincluded in storage accessible over the Internet, some components beingin wearable electronic devices or an electronic scope or an electronicsighting device or a weapon or a projectile, and some components beingin various different electronic devices that are spread across anetwork, a cloud, and/or an electronic device system or a computersystem.

The processing unit or processor (such as a central processing unit,CPU, microprocessor, application-specific integrated circuit (ASIC),etc.) controls the overall operation of memory (such as random accessmemory (RAM) for temporary data storage, read only memory (ROM) forpermanent data storage, and firmware). The processing unit or processorcommunicates with memory and performs operations and tasks thatimplement one or more blocks of the flow diagrams discussed herein. Thememory, for example, stores applications, data, programs, algorithms(including software to implement or assist in implementing exampleembodiments) and other data.

Blocks and/or methods discussed herein can be executed and/or made by auser, a user agent of a user, a software application, an electronicdevice, a computer, and/or a computer system.

Examples of an electronic device include, but are not limited to, aserver, a computer, a laptop computer, a tablet computer, a handheldportable electronic device (HPED), a portable electronic device (PED), awearable electronic device (WED), wearable electronic glasses (WEG), anelectronic scope, electronic binoculars, a smartphone, a camera, anon-portable electronic device, a movable or flyable electronic device,and an electronic device with a processor, a memory, and a display.

As used herein, “augmented reality” is a view of the real, physicalworld in which elements are augmented or modified with computer orprocessor generated input, such as sound, graphics, GPS data, video,and/or images. Virtual images and objects can be overlaid on the realworld that becomes interactive with users and digitally manipulative.

As used herein, “determine” includes to ascertain, to calculate, todecide, to obtain, to discover, to retrieve, and/or to receive.

As used herein, “field of view” or “field of vision” is the extent ofthe observable world that is seen or captured at a given moment.

As used herein, “firearm” is a portable gun, such as a rifle or apistol.

As used herein, “line of sight” is a straight line that extends from thescope, camera, or other sighting apparatus to the target.

As used herein, “naked eye” is visual perception unaided by a magnifyingdevice or a light collecting optical device.

As used herein, “orientation” is a position with relation to points of acompass or other specific directions or locations.

As used herein, “point of aim” is a visual indication of an electronicdevice that shows where the electronic device is aimed.

As used herein, “target” is one or more of a person, an object, a thing,and an area.

As used herein, “virtual image” or “virtual object” is computer orprocessor generated image or object. This image or object often appearsto a user in the real, physical world (such as a virtual 3D dimensionalobject that the user views in the real world).

As used herein, “weapon” includes firearms (such as portable guns),archery (such as bow and arrows), light weapons, heavy weapons, andother weapons that launch, fire, or release a projectile.

As used herein, “wearable electronic device” is a portable electronicdevice that is worn on or attached to a person. Examples of such devicesinclude, but are not limited to, electronic watches, electronicnecklaces, electronic clothing, head-mounted displays, electroniceyeglasses or eye wear (such as glasses in which an image is projectedthrough, shown on, or reflected off a surface), electronic contactlenses, an eyetap, handheld displays that affix to a hand or wrist orarm, and HPEDs that attach to or affix to a person.

In some example embodiments, the methods illustrated herein and data andinstructions associated therewith are stored in respective storagedevices, which are implemented as computer-readable and/ormachine-readable storage media, physical or tangible media, and/ornon-transitory storage media. These storage media include differentforms of memory including semiconductor memory devices such as DRAM, orSRAM, Erasable and Programmable Read-Only Memories (EPROMs),Electrically Erasable and Programmable Read-Only Memories (EEPROMs) andflash memories; magnetic disks such as fixed, floppy and removabledisks; other magnetic media including tape; optical media such asCompact Disks (CDs) or Digital Versatile Disks (DVDs). Note that theinstructions of the software discussed above can be provided oncomputer-readable or machine-readable storage medium, or alternatively,can be provided on multiple computer-readable or machine-readablestorage media distributed in a large system having possibly pluralnodes. Such computer-readable or machine-readable medium or media is(are) considered to be part of an article (or article of manufacture).An article or article of manufacture can refer to any manufacturedsingle component or multiple components.

Method blocks discussed herein can be automated and executed by acomputer, computer system, user agent, and/or electronic device. Theterm “automated” means controlled operation of an apparatus, system,and/or process using computers and/or mechanical/electrical deviceswithout the necessity of human intervention, observation, effort, and/ordecision.

The methods in accordance with example embodiments are provided asexamples, and examples from one method should not be construed to limitexamples from another method. Further, methods discussed withindifferent figures can be added to or exchanged with methods in otherfigures. Further yet, specific numerical data values (such as specificquantities, numbers, categories, etc.) or other specific informationshould be interpreted as illustrative for discussing exampleembodiments. Such specific information is not provided to limit exampleembodiments.

1.-20. (canceled)
 21. A method executed by one or more electronicdevices in an electronic device system, the method comprising: mapping,by the one or more electronic devices in the electronic device system,locations of objects at a geographical location; determining, by the oneor more electronic devices in the electronic device system and for oneof the objects that is an obstructed object to a view of a camera of asmartphone at the geographical location, a location and an orientationof the obstructed object that the camera of the smartphone at thegeographical location would capture if the obstructed object were notobstructed to the view of the camera; and displaying, on a display ofthe smartphone while the camera is pointed at the location of theobstructed object, a three-dimensional (3D) augmented reality (AR) imageof the obstructed object at a location on the display of the smartphonewhere the obstructed object would exist if the obstructed object werenot obstructed from the view of the camera.
 22. The method of claim 21further comprising: displaying, on the display of the smartphone whilethe camera is pointed at the location of the obstructed object, the 3DAR image of the obstructed object with an orientation that matches theorientation of the obstructed object that the display of the smartphonewould display if the obstructed object were not obstructed from the viewof the camera.
 23. The method of claim 21 further comprising: rotating,on the display of the smartphone while the camera is pointed at thelocation of the obstructed object, the 3D AR image of the obstructedobject along a Z-axis so an orientation of the 3D AR image of theobstructed object along the Z-axis matches the orientation of theobstructed object along the Z-axis that the display of the smartphonewould display if the obstructed object were not obstructed from the viewof the camera.
 24. The method of claim 21 further comprising: capturing,by the one or more electronic devices in the electronic device system,movements of the obstructed object; and moving, based on the movementscaptured of the obstructed object and on the display of the smartphonewhile the camera is pointed at the location of the obstructed object andwhile the obstructed object remains obstructed from the view of thecamera, the 3D AR image of the obstructed object to coincide with themovements of the obstructed object.
 25. The method of claim 21 furthercomprising: simultaneously displaying, while the obstructed objectremains obstructed from the view of the camera, the 3D AR image of theobstructed object on the display of the smartphone with a user and on adisplay of wearable electronic glasses (WEG) worn on a head of the user.26. The method of claim 21 further comprising: displaying, on thedisplay of the smartphone while the camera is pointed at the location ofthe obstructed object and while the obstructed object remains obstructedfrom the view of the camera, the 3D AR image of the obstructed objectalong with a 3D AR image of a frame of reference that shows an areaaround the obstructed object.
 27. The method of claim 21 furthercomprising: capturing, with a second smartphone at the geographicallocation, information about the obstructed object that includes a globalpositioning system (GPS) location of the obstructed object, an image ofthe obstructed object, and a distance from the second smartphone to theobstructed object; and sharing, with the smartphone, the informationcaptured with the second smartphone.
 28. The method of claim 21 furthercomprising: displaying, on the display of the smartphone while thecamera is pointed at the location of the obstructed object and while theobstructed object remains obstructed from the view of the camera, the 3DAR image of the obstructed object along with a distance from thesmartphone to the obstructed object, a compass direction from thesmartphone to the obstructed object, and an identity of the obstructedobject.
 29. A non-transitory computer readable storage medium storinginstructions that cause one or more electronic devices to execute amethod comprising: building, based on information captured with one ormore of smartphones and wearable electronic glasses (WEG) at ageographical location, augmented reality (AR) images of objects at thegeographical location; calculating, for one of the objects that is anobstructed object and not viewable through a camera of a smartphonewhile the camera is pointed in a direction to the obstructed object, alocation where the obstructed object would appear on a display of thesmartphone if the obstructed object were not obstructed; and displaying,on the display of the smartphone while the camera is pointed in thedirection to the obstructed object, an AR image of the obstructed objectat a location on the display of the smartphone where the obstructedobject would appear on the display of the smartphone if the obstructedobject were not obstructed.
 30. The non-transitory computer readablestorage medium of claim 29 in which the method further comprises:calculating, for the obstructed object, an orientation that theobstructed object would have on the display of the smartphone if theobstructed object were not obstructed; and displaying, on the display ofthe smartphone while the camera is pointed in the direction to theobstructed object that is obstructed to the smartphone, the AR image ofthe obstructed with the orientation that the obstructed object wouldhave on the display of the smartphone if the obstructed object were notobstructed.
 31. The non-transitory computer readable storage medium ofclaim 29 in which the method further comprises: displaying, on thedisplay of the smartphone while the camera is pointed in the directionto the obstructed object that is obstructed to the smartphone, the ARimage along with a distance from the smartphone to the obstructedobject, a compass direction from the smartphone to the obstructedobject, and an identity of the obstructed object.
 32. The non-transitorycomputer readable storage medium of claim 29 in which the method furthercomprises: tracking, with the smartphone, a gaze of a user holding thesmartphone; and automatically displaying, on the display of thesmartphone while the obstructed object is obstructed to the smartphone,a direction to the obstructed object and a distance to the obstructedobject when the gaze of the user is directed to the location on thedisplay where the AR image of the obstructed object appears.
 33. Thenon-transitory computer readable storage medium of claim 29 in which themethod further comprises: tracking, with the smartphone, a gaze of auser holding the smartphone; and automatically enlarging, on the displayof the smartphone while the obstructed object is obstructed to thesmartphone, a size of the AR image of the obstructed object when thegaze of the user is directed to the location on the display where the ARimage of the obstructed object appears.
 34. The non-transitory computerreadable storage medium of claim 29 in which the method furthercomprises: moving, on the display of the smartphone while the camera ispointed in the direction to the obstructed object and while theobstructed object is obstructed from being captured with the camera ofthe smartphone, the AR image of the obstructed object to coincide withreal-time movements of the obstructed object.
 35. The non-transitorycomputer readable storage medium of claim 29 in which the method furthercomprises: rotating, on the display of the smartphone while the camerais pointed in the direction to the obstructed object and while theobstructed object is obstructed from being captured with the camera ofthe smartphone, the AR image of the obstructed object along a Z-axis soan orientation of the AR image of the obstructed object along the Z-axismatches an orientation of the obstructed object along the Z-axis thatthe display of the smartphone would display if the obstructed objectwere not obstructed.
 36. A method comprising: building, based oninformation captured with one or more portable electronic devices at ageographical location, augmented reality (AR) images of objects at thegeographical location; determining, for one of the objects that is anobstructed object and not viewable to a camera of a smartphone while thecamera is pointed in a direction of the obstructed object, a locationwhere the obstructed object would appear on a display of the smartphoneif the obstructed object were not obstructed; and displaying, on thedisplay of the smartphone while the camera is pointed in the directionof the obstructed object, an AR image of the obstructed object at alocation on the display of the smartphone that coincides with where theobstructed object would appear on the display of the smartphone if theobstructed object were not obstructed.
 37. The method of claim 36further comprising: calculating, by one or more processors, anorientation that the obstructed object would have on the display of thesmartphone if the obstructed object were not obstructed; and displaying,on the display of the smartphone while the camera is pointed in thedirection of the obstructed object, the AR image of the obstructed withthe orientation that the obstructed object would have on the display ofthe smartphone if the obstructed object were not obstructed.
 38. Themethod of claim 36 further comprising: tracking, with the smartphone, agaze of a user holding the smartphone; and automatically displaying, onthe display of the smartphone, a direction to the obstructed object anda distance to the obstructed object when the gaze of the user isdirected to the display where the AR image of the obstructed objectappears.
 39. The method of claim 36 further comprising: determining,with the smartphone, a gaze of a user holding the smartphone; sensing,with the smartphone, a gesture command to track the obstructed objectwhen the gaze of the user is directed to the AR image of the obstructedobject; and highlighting, on the display of the smartphone and inresponse to the gesture command, the AR image of the obstructed objectto show the obstructed object is being tracked.
 40. The method of claim36 further comprising: determining, with the smartphone, a gaze of auser holding the smartphone; receiving, with the smartphone, a voicecommand to track the obstructed object when the gaze of the user isdirected to the AR image of the obstructed object; and highlighting, onthe display of the smartphone and in response to the voice command, theAR image of the obstructed object to show the obstructed object is beingtracked.